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Two  factorial  cx})erimcnts  with  possible  outliers  (John  (1978))  are 
reanalyrcd  by  rnc.-ms  of  robust  rej;rcssion  techniipics.  We  show  that  using 
M-estiaiates  of  regression  results  in  efficient  analyses  which  are  easier 
to  irnpleitient  tktn  the  inetliotLs  jiroposeJ  ’  )•  John. 
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1.  Introtluct  Lon 


In  a  recont  paper,  .lolin  (li'^8)  clLsciisseii  the  etfects  and  detection  of 
outliers  in  factorial  experiments.  His  methods  were  based  on  least  squares 
metfxxlology  and  can  he  qiiicHy  siuirnri zed  as  follows.  (Consider  the  usual 
linear  model 

Y  =  .X  B  1  . 

If  one  suspects  that  m  outliers  arc  present,  then  the  mdel  could  lx-  written 
as  111 

Y  =  .\ B  >  I  ^  '  . 

i=l 

where  d^  is  a  vector  with  1  in  tlio  rew  corrcsiiondi ng  to  the  i-th  suspected 
outlier  and  0  elstsvliere  (i  -  l,2,...,m).  The  iiresence  of  outliers  would  then 
be  tested  by  ll^ji  ^  =  0.  Ikiwever,  the  percentage  point  ^  which  one  would 
camjiare  with  the  I--statist  ic  T*  should  not  be  the  usual  upper  a  percentage 
point  of  tlic  1-  ilistribut ion  since  tlie  observations  being  tested  have  the  most 
extreme  residuals. 


,lohn  then  discusses  a  simple  modification  when  tliere  is  only  one  suspected 
outlier  (m  =  1  —  use  the  ujiper  i/N  percentage  point  of  the  F-distribution) . 

For  two  suspected  outliers  (m  =  2),  a  fairly  deep  analysis  is  rcciuired,  including 
tlie  use  of  simulation.  For  in  7i,  the  method  becomes  even  more  detailed. 

At  best,  the  tyjie  of  cuialysis  presented  alxive  is  messy,  time -consuming  and 
conplcx.  riic  method  of  detennining  the  percentage  point  F^  is  neither  unified 
nor  written  as  a  sin^ile  algorithm,  ;md  outlier  rejection  itself  requires  skill 


and  care.  Practicing  statisticians  with  deadlines  to  meet  will  likely  not  have 
the  time,  inclination  nor  energy  to  cto  such  an  analysis,  while  the  conmon  naive 
asers  of  canned  statistical  prognims  (such  as  SPSS)  will  not  have  the  technical 


comiietance  to  do  the  analysis, 
outliers. 


The  basic  diti'icultv  is  tliat  least  sunaiis  is  sensitive  to  outliers,  v^hicli 
can  drastically  chaiiiie  I'aiiuaetci  estinsites  as  well  as  disyuise  significant 
effects;  tliis  is  ;uni)ly  illustrated  by  .lolji's  first  exaiiple  (sec  Section  3  below). 
In  oivler  to  improve  the  iiuality  of  thi  nviss  of  statistical  analyses,  wiuit  is 
needed  are  not  co!^ilex,  /s  •  procedures  for  iletect  ine  and  modifying  outliers, 
blit  i.ith.er  pnvedures  wliich  are  insensitive  to  the  jiresence  of  outliors  (see 
Cook  n‘d77j  for  a  treatment  ol  outliers  in  independent  variables  in  regression). 
Ideally,  such  robust  (non- least  s(|uares)  methods  dv'iild  give  results  (for  most 
problems!)  whicli  are  siinilar  to  those  obtaincxl  !>)■  outlier  detection  and  ;:iodifi- 
caticn  procedures.  iTie  reci-nt  statistical  literature  alx)unds  with  proposals 
to  attain  this  goal,  the  most  jxipular  of  which  are  Huber's  (1PG4,  1973,  1977) 
M-est  iiiiates.  This  rather  intensively  studied  class  has  been  specifically 
designed  to  lx;  inseasitive  to  outliers  aixl  to  retain  high  efficiency  when  the 
errors  are  heavier- tai  led  tlian  the  noniual,  two  properties  not  possessed  by 
least  squares.  Hie  basics  of  M-est imates  are  reviewed  in  Section  2.  In 
St-  tion  3  we  apply  these  iiK'thods  to  .John's  first  example,  while  in  Section  4 
we  discuss  dolui's  second  example.  We  find  in  both  cases  that  one  jiass  througli 
a  robust  regression  program  based  on  M-est invites  yields  results  closelx  similar 
to  those  obtained  by  John's  coiniilex  analysis. 

rbniel  and  Wood  (1971)  and  Andrews  (1974)  analyze  a  ilata  set  in  a  regression 
context.  Tlie  fonner  use  least  siiuares  and,  decide  (after  very  detailed  analysis) 
that  there  are  three  outliers.  /\ndrews  uses  robust  techniques  similar  to  those 
presented  here  and  sliows  that  tlv’  decision  favoring  three  outliers  can  be  reacJied 
in  a  much  easier  and  more  routine  fashion,  Hius,  the  advantages  of  robust 
tecliniques  are  not  limited  to  the  factorial  experiiiK'iits  aiuilyzed  herein. 
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2.  M-cst  iiiiatos  of  Regression 

1-east  scuLircs  estinutes  minimise 

(2.1)  I  ,)f(y.-x.B)/n)  , 

i=l  ^ 

-7 

wliere  pfx)  -  ‘jX  .  ITie  i|iiai!ratic  lorm  of  (  is  w)iat  nuikes  least  squares  sensitive 
to  outliers,  iliis  eaii  al^o  he  seen  if  one  ilel  iiios  q  =  o',  for  then  one  solves 


(2.2) 


)>((>■; -X- B)A-)X.  =  h  , 
i=l  '  ‘  ^ 


where  for  least  squares  4/(Z)  =  Z.  In  order  to  achieve  robustness  against  outliers 
and  high  efficiency  for  distrihutions  heavier- t ai led  than  the  normal,  fluber 
(1964),  .•\ndrcws,  et  al  (1972)  and  ILanjicl  (1974)  suggest  that  be  a  bounded 
function,  arid  that  scale  l)e  estimated  in  one  of  two  ways; 

(Proposal  2)  Solve  simultaneously  (2.2)  and 

(2.3)  (n-p)  ^)^iJ'^((yj-x.B)/o)  =  li4^^(Z)  , 


the  expectation  being  under  the  staiubn!  noniuil. 


(2.4) 


(M\I))a 


median  absolute  residiuil 
from  median  I 


/.6745 


(Tins  is  asyn^)totical  ly  cxiual  to  one  foi  the  nonnal  model.) 

In  both  cases,  tlie  .solutioti  is  foiuid  iteratively,  tine  chooses  a  starting  value 
for  •,  solves  (2.2),  t)ie:i  ululates  by  (2.3)  or  (2.4),  etc.,  continuing  until 
convergence.  Algoritluiis  ari’  avail.ible  in  lluUjr  (1973,  1977)  and  Dutter  (1976); 
the  author  has  adapted  tliese  algorithms  for  use  in  the  SAS  computer  programs 
(a  card  deck  is  available  upon  reejuest).  In  neither  case  is  the  computation 
burdensome . 

The  tyi)ical  choices  of  ip  are 

llulx?r*s  i;/(x)  =  -V'(-x)  *  max(-k,  min(x,k))  ,  with  k  generally  taken 
as  1.6  or  2. (I. 
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H;um>ci'  s 


(c-x) 


0  ‘  X  •  a 


a  X  >  b 


I)  •  X  •  c 


Andrews ' 


Kx)  =  -4'(-x) 


X  c  . 


sinefx/c)  0  •  x  ■  t.  n 


X  ;•  ct: 


Tlie  constant  c  is  often  taken  as  2.1. 


The  fact  that  ltimi)el's  ip  and  /\ndrcws'  ij;  botli  redescend  to  zero  suggests 
(llanpcl  (1974))  that  they  giw  no  weight  to  gross  outliers,  while  Huber's  p 
will  give  some  weight  to  these  outliers  but  not  nearly  so  much  as  least  squares. 
The  redescending  ij;  functions  (unlike  Uibor's  p)  can  have  problems  with  conver¬ 
gence;  for  this  reason  we  adojit  the  convention  of  first  estimating  g  by  itiber’s 
metliod  and  then  using  at  nv  .t  two  iterations  of  the  algoritlim  for  ilan^jcl's  and 
■Andrews'  methods,  in  all  these  cases,  urhler  proper  conditions,  the  robust 
regression  estimate  of  g  is  asNiriptotical ly  normally  distributed  with  mean 
B  .ind  co\.iriancc  matrix  whicii  can  be  estimated  by 


(2.4) 


(no");;^r.)/([^'(r.)}^)CX'X)'^  , 


wlx;re  the  stajidardi zed  residuals  are 


Huber  (1973)  and  Andrews,  et  al  (1972)  show  in  simulation  exj^'riments  that 

A 

t'ne  estimates  are  generally  more  efficient  than  the  least  squares  estimates, 
they  arc  only  slightly  more  variable  than  least  sepurcs  for  the  nomvil  iiK'del  Imji 
are  considerably  less  variable  for  heavier-tailed  iiKxlels. 


Inference  alx)ut  tlu‘  pai;uni'U'is  can  take  at  least  two  forms,  Injth  based  on 
the  ap^iroximatiun  (2.4).  Schrader  atid  llet tm;msf)erj;cr  (uiipiil)lished)  suggest  an 
analysis  of  the  usu;il  drop  in  stun  of  sqitircs  statistic  using  5^p(r^).  Bickel 
(1976,  discussion  section)  suggests  the  approach  used  hero.  lx‘t 

-1 

/  =  n  ).  (r:  ) 

i  =  l 

()  =  1  ♦  (p/n)(l  A)/>  . 

Ihe  term  ri  is  suggested  by  Ihilu'r  (1973,  C'liiation  )  as  a  variance  infl.ation 
factor  for  (2.4)  if  p/n  is  not  snuill.  IVfine  :  .••■k.Ii  -lyi '  ic  r 

"  -i‘Sl  ^  'ioi)<(r.)/\  . 

Then,  the  least  sqiurcs  est mutes  for  the  nKxiel  Y  =  £  ^re  exactly 

(this  follows  from  (2.2)1.  Rickcl  suggests  that  asymptotically  correct  tests 
c.in  be  obtained  In’  defining  the  pseixIo-valiK's  ;ind  using  them  in  conventional 
least  sciuares  packages. 

In  the  exainjilcs  below,  wt'  used  a  givc'ii  b)’  (2.3);  for  lluber's  ip  we  took 
k  =  1.5,  for  ll;rii['crs  a  =  1..5,  b  =  .3.5,  c  =  R.O,  while  for  ;\ndrows',  c  =  2.1. 


3.  First  i.xaiiiple 

•1-1  J 

■John’s  first  (‘x.amph'  is  ;i  .3  fractinaal  rejtl  icat  ion  of  a  3  experiment, 
fhe  effects  of  each  factor  are  split  into  linc.ar  and  quadratic  conponents 
(AI,,  AQ,  IM,,  IK),  CL,  (IQ,  HI.,  IM))  ;uul  three  interactions  are  formed  by  multipli¬ 


cation  (ALBI,,  AU’.h,  R1£I,).  (tbservat ion  11  is  a  suspected  outlier;  the  predicted 
values  ajid  residuals  for  the  four  methods  are  giveti  in  Table  1.  In  Figure  1 
we  present  schematic  i)lots  of  the  residuttls  for  tlie  four  metliods  (sec  Tukey 


(19'^2)).  The  length  of  the  Ikjx  corresponds  to  the  interquartile  r;mge,  the 


length  of  the  tails  is  described  by  the  vcrticle  dashed  lines,  and  potentially 


serious  outliers  are  indicated  by  the  synilxjl  fhe  obvious  conclusion  from 

Table  I  is  that  the  Ituipel  and  /\ndrews  ntethod  are  particularly  robust  in  that 
their  predicted  value  for  observation  11 
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(il  Is  close  to  .Jolui's  v.iliio; 

(ii)  hardly  ohaiiy,c‘s  whoii  the  oiigiiial  obsojvati(jn  y  -  It  is  replaced  by 
the  rel  itted  valiK*  y  = 

I-rom  Figure  1  we  see  tluit  tlie  u)!nist  methods  fit  the  chita  much  better  than 
does  least  squires,  again  uidicat  iiig  the  value  of  (I.  e  pt'K'ediites. 

In  Table  2  we  ['resent  s igni ric.ince  levels  I'or  the  cffeets  iisifig  tiie 
original  observations  ;ind  then  using  the  refitted  observation  11.  Tlie  striking 
features  are  that 

(i)  Tlie  IkimiH’l  and  ,\ndiew;.  methods  give  in  one  jiass  on  the  orig.inal 
observations  essentially  the  same  anahsis  as  does  Jelin's  Siore 
complex  refitted  data; 

fii)  the  signi f iciuioe  levels  of  the  ILuiipel  and  iXndrcws  methods  do  not 
change  to  any  large  extent  after  oliservation  11  is  modified. 

4- 1 

We  conclude  that  for  the  ?>  exaiiijile,  the  robust  regression  methods  con^iare 
favorably  with  Jolin's  metiuxl.  Ihe  signif iciuice  levels  arc  similar  and  the 
robust  methods  aie,  in  general,  considerably  easier  to  imjilemcnt. 

4 .  Second  l-xtimple 

Tlie  second  exam]ile  Jolui  uses  to  illustrate  his  method  is  a  confoiiiidcd  2^ 
experiment,  the  block  effects  confounding  the  higliest  order  interaction.  After 
lengthy  analysis  he  concliules  that  two  susjicctcd  outliers  arc  not  really  outliers 
and  should  not  be  refitted.  In  Table  3  we  present  significance  levels  for  the 
four  tests,  while  in  Figure  2  the  seheiivitie  plot  of  the  lesidtials  is  givcii. 

Tlic  tiuijor  difference  between  least  squares  and  the  robust  estimates  exhibited 
in  Table  3  is  that  the  latter  show  a  main  effect  in  B  significant  at  tlie  .05 
level,  while  for  least  squares  the  significance  level  is  approximately  .13. 
Although  Jolin's  analysis  suggests  that  there  may  well  he  no  jroaa  outliers,  we 
sec  that  tlie  treatment  combinations  ad  and  d  are  suf  ficicnl  ly  discrcp:int  from 
the  others  so  as  to  inflate  the  least  squires  mean  square  error  and  thus 
obscure  what  appears  to  be  a  significant  main  effect.  The  IL'unpcl  and  Andrews 
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IakIs  also  1  inJ  titi'  HI!  iiil oi  k  t  ion  to  I''’  fuKvIoi'afol y  )  s i (mi i  (  irarit . 

In  the  prcvioiLs  section  v*c  louiul  that  a  y.ross  ootlicr  can  radically 
aftect  a  least  squares  analysis,  while  hivini;  a  pinch  spviller  effect  on  the 
robust  ninhods.  In  this  exainik  we  have  sei'ii  that  sliv.htly  discrqiant  obser¬ 
vations  fperhqas  due  to  a  distribution  heavier-tai led  than  the  normal)  can 
inf  late  the  least  s(|uares  int'an  scjuarc  error,  causing  it  to  lose  efficiency. 


5.  Discussion 

The  exan^iles  made  clear  that  tlicre  is  miKh  to  be  i',ained  by  usiny,  M-estiiivite^ 
of  regression.  Tlie  treatment  of  designs  (with  i>ossihle  outliers)  by  these 
methods  is  efficient  luul  easy  to  implement.  The  (juality  of  statistical  analyses 
will  lx*  greatly  ipi/noved  by  routine  ase  of  M-estimates  as  one  of  the  statistician's 
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Table  1 
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Pietlictcd  values  and  residuals  for  observation  11  ia  the  3  experi- 


meat,  based  on  the  original 

ohservaticMis. 

.Jo!  ji '  s 

refitted  value 

is 

t)2.33. 

Least  SlT'iaies 

llulier 

Ituiipel 

Ai  id  rows 

Of) served  value 

n 

14 

14 

14 

Predicted  valm* 

46.2 

53.7 

57.7 

59.4 

Residual 

-32.2 

-41.7 

■43.7 

-45.4 

Refitted  value 

62.33 

62.33 

62.33 

62.33 

I’rediLtid  value  after  refittin,;; 

62.33 

61.2 

61.2 

61 .6 

Residual  after  refitting 

n 

1.1 

1.1 

0.7 

Significance  levels  for  tl'.e  3^  ^  »*\|)erirK“nt  using  the  four  methods, 
hlank  values  indicati'  a  significance  level  greater  than  O.UJ. 
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Tabu-  3 


Significance  levels  ior  the  coiiUniiulnl  2^  experiment.  Rlank.s 
indicate  a  level  greater  than  0.10. 


Source 

Ix-ast  .‘■xpi.ires 

1  IuIk-i 
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Aiulrev%s 

A 

.01 

.00 

.00 

.00 
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C 

.O.'S 

.01 

.01 

.0.3 
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Schc.TJtic  plots  of  residuals  for  the  four  rtthoJs  in  the  confounded  I  experi»-T 
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